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AN  ALGORITHM  FOR  COMPUTERIZED  ADAPTIVE  DECISION  ANALYSIS 


BRIEF 


Requirement : 

To  develop  new  methods  for  analysis  of  decision-making  behavior — 
specifically,  new  methods  for  (1)  modeling  a  decision  process  used  to 
evaluate  preferences  for  complex  choice  alternatives,  and  (2)  producing 
measurement  scales  for  choice  component  factors  and  composite  choice 
alternatives  based  on  the  decision  model. 


Procedure : 

Recent  results  in  conjoint  measurement  theory  research  were  applied 
to  develop  an  algorithm  for  minimizing  the  problems  of  redundancy  and 
random  error  in  testing  the  additive-independence  model  in  pair-comparisons 
designs  with  fallible  data.  No  provisions  were  made  for  handling  the 
systematic  error  problem  or  for  accommodating  more  than  two  choice  com¬ 
ponent  factors. 


Findings : 

The  algorithm  was  tested  in  error-free  data  and  in  data  with  random 
error.  A  reduction  of  approximately  44%  in  the  number  of  pair-comparisons 
necessary  to  determine  all  model  constraints  was  obtained  in  a  5  x  5  fac¬ 
torial  design  with  error-free  data.  This  reduction  was  decreased  to  33% 
when  a  moderate  amount  of  random  error  was  introduced.  Various  deficien¬ 
cies  in  algorithm  performance  were  noted.  In  general,  the  results  showed 
that  this  algorithm  performed  rather  poorly. 


Utilization  of  Findings: 

This  research  demonstrates  some  limitations  of  a  direct  application 
of  the  generalized  cancellation  condition  for  testing  the  additive- 
independence  model.  Further  research  is  required  to  develop  an  analyti¬ 
cal  approach  which  can  ameliorate  these  limitations. 
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AN  ALGORITHM  FOR  COMPUTERIZED  ADAPTIVE  DECISION  ANALYSIS 


INTRODUCTION 

General  Statement  of  the  Problem 


In  Army  research  it  is  often  desirable  to  investigate  the  nature  of 
a  decision  process  used  by  Army  personnel  in  making  some  important  choice. 
In  many  instances  the  choice  alternatives  may  represent  composites  of 
several  component  factors  wh'^ch  may  be  identified  and  considered  sepa¬ 
rately.  For  example,  in  Army  career  planning  research  it  may  be  of  in¬ 
terest  to  investigate  the  nature  of  the  decision  process  used  by  officers 
in  indicating  their  preferences  for  various  assignments.  In  this  ex¬ 
ample  the  choice  alternatives  may  be  described  as  composites  of  component 
factors  such  as  assignment  location  (e.g.,  EUROPE,  CONUS),  type  (e.g.  , 
COMMAND,  STAFF),  and  duty  specialty  (i.e.,  PRIMARY,  ALTERNATE). 

The  additive- independence  model  (AIM)  is  one  partict  ar  model  of  the 
decision  process  which  has  considerable  appeal  because  of  its  simplicity 
in  comparison  with  other  possible  models  and  because  of  its  general  ap¬ 
plicability  in  other  substantive  areas  of  psychology.  In  this  model 
component  factor  levels  are  assigned  specific  scale  values  relative  to 
one  another.  The  scale  values  assigned  to  levels  of  a  component  factor 
are  designated  independently  for  each  component  factor.  The  scale  value 
attached  to  a  particular  composite  choice  alternative  is  determined  by 
summing  the  scale  values  assigned  to  levels  of  the  component  factors 
which  are  present  in  the  composite.  The  relative  magnitude  of  scale 
values  for  composite  choice  alternatives  serves  as  a  basis  for  the  pref¬ 
erence  decision. 

t 

For  example,  EUROPE  =  1  and  CONUS  =  2  may  be  the  scale  values  as¬ 
signed  to  assignment  location;  COMMAND  =  2  and  STAFF  =  3  may  be  the  scale 
values  assigned  to  assignment  type;  and  PRIMARY  =  5  and  ALTERNATE  =  1  may 
be  the  scale  values  assigned  to  assignment  duty  specialty.  If  an  indi¬ 
vidual  uses  the  AIM  in  evaluating  preferences  for  various  assignments, 
the  scale  value  attached  to  a  PRIMARY  specialty  COMMAND  assignment  in 
EUROPE  is  5  +  2  +  1  =  8;  the  scale  value  attached  to  a  PRIMARY  specialty 
STAFF  assignment  in  CONUS  is  5  +  3  +  2  =  10.  Since  the  composite  scale 
value  for  the  latter  choice  alternative  exceeds  the  composite  scale  value 
for  the  former  choice  alternative,  the  individual  would  indicate  a  pref¬ 
erence  for  the  latter  choice  alternative.  If  composite  scale  values  for 
the  choice  alternatives  are  equal,  an  individual  may  indicate  no 
preference . 

In  research  on  the  nature  of  a  decision  process  two  specific  objec¬ 
tives  may  be  identified.  The  first  objective  is  to  develop  a  model  of 
the  decision  process  used  to  evaluate  preferences  for  complex  choice 
alternatives.  If  individual  differences  in  the  decision  process  are 
discovered,  an  individual-specific  model  of  the  decisicn  process  may  be 
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required.  The  second  objective  is  to  produce  measurement  scales  for 
choice  component  factors  and  composite  choice  alternatives  based  on  the 
decision  model.  The  general  purpose  of  this  study  was  to  investigate 
an  application  of  recent  results  in  conjoint  measurement  theory  research 
with  the  aim  of  developing  a  new  methodology  for  accomplishing  these  two 
research  objectives. 


Specific  Statement  of  the  Problem 

The  conjoint  measurement  problem  is  that  of  obtaining  measurement 
scales  for  component  stimuli  and  composite  stimuli  simultaneously  based 
on  a  specified  composition  model  when  only  the  rank  order  of  the  com¬ 
posite  stimulus  effects  is  known.  The  order  constraints  generate  a 
finite  system  of  homogeneous  equations  and  inequalities  in  the  composi¬ 
tion  model.  If  the  specified  composition  model  is  valid,  the  resulting 
system  should  be  consistent  and  measurement  scales  for  component  stimuli 
and  composite  stimuli  may  be  derived  by  solving  the  system  for  the  un¬ 
known  parameters.  In  the  case  of  the  AIM,  the  derived  measurement  scales 
constitute  "multidimensional  ordered-metric  scales"  (Krantz,  Luce,  Suppes, 
and  Tversky,  1971,  Chapter  9). 

Although  conjoint  measurement  theory  has  been  particularly  well- 
developed  for  the  additive-independence  composition  model,  practical  ap¬ 
plications  with  individual  subjects  have  been  few.  Three  formidable 
problems  remain  to  be  solved  for  applications  of  conjoint  measurement 
theory  in  fallible  ordinal  data.  First,  the  magnitude  of  the  pair- 
comparison  task  may  exceed  the  capability  of  the  individual  subject  when 
the  number  of  composite  stimuli  is  large.  If  the  composite  stimulus  set 
contains  N  elements,  N(N-l)/2  nontrivial  pair-comparisons  must  be  made 
to  provide  a  complete  ordering  of  the  composite  stimuli.  Since  the  num¬ 
ber  of  pair-comparisons  usually  greatly  exceeds  the  number  of  parameters 
to  be  estimated  for  the  composition  model,  it  is  clear  that  a  large  amount 
of  redundancy  of  effort  is  inherent  in  the  complete  pair-comparisons 
method.  Second,  the  presence  of  random  experimental  error  may  introduce 
ordinal  inversions  in  the  ordering  of  composite  stimuli.  The  result  is 
an  inconsistent  system  of  equations  and  inequalities  which  provides  only 
an  approximation  of  the  latent  measurement  structure.  Third,  the  possi¬ 
ble  presence  of  systematic  error  may  be  difficult  to  detect  in  the  pres¬ 
ence  of  random  error.  Systematic  error  may  result  when  the  latent  compo¬ 
sition  model  differs  from  the  additive  model. 

The  specific  purpose  of  this  study  was  to  develop  an  algorithm  for 
interactive  conjoint  measurement  (ICM)  to  minimize  the  problems  of  re¬ 
dundancy  and  random  error  for  applications  of  additive  conjoint  measure¬ 
ment  theory  in  fallible  ordinal  data.  Although  the  algorithm  presently 
has  no  provision  for  handling  the  systematic  error  problem  and  is  limited 
to  two  component  factors,  extensions  of  the  algorithm  to  test  several 
models  simultaneously  with  multifactor  composite  stimuli  may  be  possible. 
In  the  next  section  the  conjoint  measurement-theoretical  basis  for  the 
algorithm  is  presented.  The  notion  of  a  constraints  matrix  is  defined 
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and  an  ICM  algorithm  is  described.  An  evaluation  of  the  performance  of 
the  algorithm  is  provided  in  the  last  section. 


INTERACTIVE  CONJOINT  MEASUREMENT 
Measurement-Theoretical  Basis 


Let  us  represent  component  stimulus  variables  A  and  P  as  the  finite 
sets  {a,b,c,...}  and  {p,q,r,...},  respectively,  where  the  elements  of  a 
set  are  the  particular  component  stimuli  included  in  an  experiment.  Each 
composite  stimulus  may  be  represented  as  an  element  in  the  Cartesian 
product  set  A  x  P.  For  example,  composite  stimulus  (a,p)  represents  the 
combination  of  stimulus  component  a  in  A  with  stimulus  component  p  in  P. 
Suppose  all  possible  pairs  of  composite  stimuli  are  presented  to  a  sub¬ 
ject  whose  task  is  to  indicate  the  preferred  composite  stimulus  in  each 
pair.  The  experiment  gives  rise  to  a  binary  preference  relation  (de¬ 
noted  £)  on  A  x  P.  The  two  sets  of  component  stimuli  A  and  P  together 
with  the  binary  relation  £  defined  on  A  x  P  constitute  an  empirical  re¬ 
lational  structure  (denoted  <A,  P . 

The  empirical  relational  structure  Ca,  P,^>may  be  said  to  satisfy 
the  additive-independence  model  (AIM)  if  there  exist  real-valued  com¬ 
ponent  stimulus-scale  functions  4>a  on  A  and  <f> p  on  p  such  that  the  addi¬ 
tive  combination  of  the  component  stimulus-scale  values  preserves  at 
least  the  rank  ordering  of  the  composite  stimuli  for  all  a,  b  in  A  and 
for  all  p,  q  in  P: 

(1)  (a,p)  K  (b,q)  iff  cf)  (a)  +  <j>  (p)  -  <j>  (b)  +  <f>  (q)  . 

A  p  A  p 

The  symbol  "iff"  is  an  abbreviation  for  "if  and  only  if."  Component 
stimulus-scale  functions  <j>A  and  tj>p  map  elements  in  sets  A  and  P,  re¬ 
spectively,  into  the  set  of  real  numbers  (denoted  Re) .  When  such  homo- 
morphisms  exist,  the  empirical  relational  structure  <A,  P,  is  said 
to  be  mapped  into  the  numerical  relational  structure  <J*e,  Re,  in  the 
sense  that  <})A  maps  A  into  Re,  4>p  maps  P  into  Re,  and  K  is  mapped  into  i 
defined  appropriately  on  Re  x  Re.  In  practice,  the  functions  <j>A  and 
<}>p  may  be  constructed  by  solving  the  finite  system  of  homogeneous  linear 
equations  and  inequalities  generated  by  %  on  A  x  P. 

In  order  for  solutions  $A  and  to  exist,  Scott  (1964)  proved  that 
the  following  conditions  were  necessary  and  sufficient: 

Connectedness.  Either  (a,p)  £  (b,q)  or  (b,q)  %  (a,p)  for  all  a,b 
in  A  and  for  all  p,  q,  in  P. 

Cancellation .  For  all  sequences  Sq ,  ap,  ...,  an  in  A,  for  all 
sequences  pq ,  pp,  ...,  pn  in  P,  and  for  all  permutations  n  and^o  of 
{0,  1,  ...,  n},  where  n  >  0,  if  for  i  =  1,  2,  ...,  n  (a^,  pj_)  % 

(an(i)'Pa(i))'  th6n  (\(0)'Pa(0))  *  (a0'  P0)  * 


The  connectedness  axiom  simply  requires  that  all  composite  stimuli 
must  be  comparable.  Although  the  connectedness  axiom  may  be  assumed  to 
hold  trivially  in  many  experimental  applications,  its  validity  may  be 
questioned  in  others  (Tversky,  1967;  Krantz  et  al.,  1971,  p.  17).  A 
more  general  axiomatization  of  the  AIM  which  does  not  require  connected¬ 
ness  is  presented  by  Tversky  (1967).  For  the  purposes  of  this  study, 
the  axiom  is  assumed  to  hold  in  the  latent  psychological  composition 
process. 

Since  a  sequence  of  elements  in  a  set  may  contain  repetitions  of 
elements  (as  opposed  to  a  subset  of  elements  which  may  not  contain  repe¬ 
titions)  ,  the  cancellation  axiom  actually  defines  a  countably  infinite 
set  of  cancellation  axioms  indexed  by  n.  Each  nth-order  cancellation 
axiom  asserts  that  n_  inequalities  imply  an  additional  inequality  via 
the  AIM  provided  that  identical  terms  may  be  canceled  from  each  side  of 
the  inequalities  until  only  one  term  from  each  component  stimulus  set 
remains  on  each  side  (Krantz  et  al. ,  1971,  p.  427).  Thus,  if  the  nth- 
order  cancellation  axiom  fails  from  some  n,  than  a  fortiori  the  mth-order 
cancellation  axioms  must  also  fail  for  all  m  >  n.  Similarly,  if  the 
nth-order  cancellation  axiom  holds  for  some  r±,  then  a  fortiori  the  mth- 
order  cancellation  axioms  must  hold  for  all  m  <  n_. 

The  I CM  algorithm  used  in  this  study  was  based  on  testing  successive 

cancellation  axioms  (i.e.,  n  =  1,  2,  _ ).  The  algorithm  minimized  the 

redundancy  problem  by  selecting  for  presentation  to  the  subject  only  those 
composite  stimulus  pairs  which  are  critical  for  determining  AIM  constraints. 
All  other  (redundant)  constraints  were  derived  via  the  AIM  from  knowledge 
of  previously  obtained  ordinal  constraints.  The  algorithm  minimized  the 
random  error  problem  by  detecting  ordinal  inconsistencies  in  real-time 
and  attempting  to  rectify  the  discrepancies  by  repeating  critically  im¬ 
portant  pair-comparisons.  An  extension  of  the  ICM  algorithm  based  on 
Tversky 's  (1967)  irref lexivity  axiom  may  be  possible  for  testing  multi¬ 
factor  polynomial  composition  models  of  specified  degree. 


Constraints  Matrix 


A  central  concept  in  the  development  of  the  ICM  algorithm  is  the 
notion  of  a  constraints  matrix  which  indicates  the  preference  relation¬ 
ships  among  composite  choice  alternatives.  A  constraints  matrix  de¬ 
scribes  the  empirical  binary  preference  relation  defined  over  the  Car¬ 
tesian  product-set  A  x  P.  If  the  row  composite  stimulus  is  preferred  to 
the  column  composite  stimulus,  enter  "+"  in  the  matrix.  If  the  column 
composite  stimulus  is  preferred  to  the  row  composite  stimulus,  enter 
in  the  matrix.  If  no  preference  is  indicated  between  the  two  com¬ 
posite  stimuli,  enter  "0"  in  the  matrix.  A  constraints  matrix  is  skew- 
symmetric  with  zeroes  along  the  main  diagonal.  Thus,  the  matrix  contains 
M  =  N(N-l)/2  nontrivial  terms,  where  N  is  the  number  of  elements  in  the 
Cartesian  product-set.  A  total  of  constraints  matrices  may  exist  for 
a  specified  Cartesian  product-set.  Some  of  these  matrices  may  be  fit 
with  an  additive  model,  while  others  may  not. 
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In  order  to  illustrate  the  concept  of  a  constraints  matrix,  let  us 
consider  the  simplest  case  of  a  2  x  2  factorial  design.  The  following 
two  sets  of  levels  of  component  factors  may  be  considered: 

Assignment  =  {a,  b]  =  {COMMAND,  STAFF] 


Place  =  {p,  q}  =  {EUROPE,  CONUS). 


The  corresponding  Cartesian  product-set  is  given  by 


where 


AxP  =  { (a ,p) ,  <a,q),  (b,p) ,  (b,q) } , 


(a,p)  =  ap  =  COMMAND  assignment  in  EUROPE, 

(a,q)  =  aq  =  COMMAND  assignment  in  CONUS, 

(b,p)  =  bp  =  STAFF  assignment  in  EUROPE, 

(b,q)  =  bq  =  STAFF  assignment  in  CONUS. 

An  example  of  a  constraints  matrix  which  may  be  fit  with  an  additive 
model  is  shown  in  Table  1.  The  finite  system  of  homogeneous  linear  equa¬ 
tions  and  inequalities  determined  by  this  constraints  matrix  is: 


a  +  p 
a  +  p 
a  +  p 
a  +  q 
a  +  q 
b  +  p 


q  <  0  or  p 
p  >  0  or  a 
q  >  0  or  a 
p  >  0  or  a 
q  >  0  or  a 
q  <  0  or  p 


q  <  0 
b  >  0 
b  >  q  ■ 
b  >  p  ■ 
b  >  0 
q  <  0. 


This  system  may  be  solved  for  component  scale  values  a,  b,  p,  and  q 
which  constitute  "ordered-metric"  scales  (i.e.,  the  resulting  scales 
have  properties  better  than  mere  ordinal  scales  but  not  so  good  as  in¬ 
terval  scales) .  Note  the  redundancy  of  the  equations  and  inequalities 
in  the  system.  Only  the  first  three  constraints  are  necessary  to  com¬ 
pletely  describe  the  system. 

An  example  of  a  constraints  matrix  which  may  not  be  fit  with  an 
additive  model  is  shown  in  Table  2.  The  finite  system  of  homogeneous 
linear  equations  and  inequalities  determined  by  the  constraints  matrix 


a+p-a-q<0orp-q<0 

a+p-b-p<0ora-b<0 

a+p-b-q<0ora-b<q-p 

a+q-b-p>0ora-b>p-q 


\  . '-t* 


a  +  q-  b-  q>0ora-b>0 
b+p-b-q<0orp-q<0. 


Note  the  inconsistency  of  the  equations  and  inequalities  in  the  system. 
This  system  may  not  be  solved  for  component  scale  values  which  satisfy 
the  AIM. 


Table  1 

Additive  Constraints  Matrix 


ap 

aq 

bp 

bq 

ap 

0 

- 

+ 

+ 

aq 

+ 

0 

+ 

+ 

bp 

- 

- 

V 

0 

- 

bq 

- 

- 

+ 

0 

Table  2 

Nonadditive  Constraints 

Matrix 

ap  aq 

bp 

bq 

ap 

0 

- 

- 

aq 

+  0 

+ 

+ 

bp 

+ 

0 

- 

bq 

+ 

+ 

0 

Algorithm  Logic  j 

i 

i 

The  objectives  of  the  ICM  algorithm  are  twofold;  (1)  to  minimize 
the  number  of  comparisons  between  choice  alternatives  required  to  generate 
a  complete  constraints  matrix  for  the  AIM;  and  (2)  to  rectify  discrepan¬ 
cies  from  the  AIM  due  to  random  error.  A  method  of  accomplishing  these 
objectives  will  be  presented  below. 


The  I CM  algorithm  consists  of  three  successive  stages.  In  the  first 
stage  the  first-order  cancellation  axiom  is  used  to  construct  a  con¬ 
straints  matrix  that  satisfies  the  first-order  cancellation  properties 
of  the  AIM.  In  the  second  stage  the  second-order  cancellation  axiom  is 
used  to  construct  a  constraints  matrix  that  satisfies  the  second-order 
cancellation  properties  of  the  AIM  as  well  as  the  first-order  cancella¬ 
tion  properties.  In  the  third  stage  the  resulting  constraints  matrix 
may  be  passed  to  a  linear  programming  subroutine  which  tests  the  con¬ 
straints  matrix  for  additivity.  If  a  solution  exists,  the  subroutine 
may  generate  scale  values  for  the  component  factors  of  the  composite  choice 
alternatives  based  on  the  AIM. 

First-order  cancellation  conditions  consist  of  two  forms  of  single¬ 
component-wise  cancellation.  The  A-component-wise  cancellation  form  is: 

If  (a,p)  £  (a,q) , 

then  (b,p)  £  (b,q) 

for  all  a,  b  in  A  and  for  all  p,  q,  in  P.  The  P-component-wise  cancella¬ 
tion  form  is: 

If  (a,p)  £  (b,p) , 

then  (a ,q)  %  (b,q) 

for  all  a,  b,  in  A  and  for  all  p,  q  in  P.  Since  the  antecedent  inequality 
implies  the  conclusion  inequality,  only  the  antecedent  constraint  must  be 
ascertained  by  querying  the  subject.  The  conclusion  constraint  may  be 
derived  since  first-order  cancellation  is  a  necessary  condition  for  the 
AIM. 

Second-order  cancellation  conditions  consist  of  two  forms  of  double- 
component-wise  cancellation.  In  the  simple  transitivity  form,  the  A- 
component  and  P-component  of  one  composite  stimulus  in  each  of  the  two 
antecedents  are  identical: 

If  (a ,p)  %  (b,q) 

and  (b,q)  £  (c,r)  , 

then  (a,p)  £  (c,r) 

for  all  a,  b,  c,  in  A  and  for  all  p,  q,  r  in  P.  In  the  generalized 

transitivity  form,  the  A-component  and  P-component  of  each  composite 
stimulus  in  the  antecedents  are  different: 


i 


If  (a,p)  £  (b,q) 


and  (b,r)  £  (c,p), 
then  (a , r)  £  (c,q) 

for  all  a,  b,  c  in  A  and  for  all  p,  q,  r  in  P.  Since  the  antecedent  in¬ 
equalities  imply  the  conclusion  inequality,  only  the  antecedent  constraints 
must  be  ascertained  by  querying  the  subject.  The  conclusion  constraint 
may  be  derived  since  second-order  cancellation  is  a  necessary  condition 
for  the  AIM. 

In  the  general  case  first-  and  second-order  cancellation  properties 
are  necessary  but  not  sufficient  for  a  constraints  matrix  to  satisfy  the 
AIM  since  higher-order  cancellation  properties  which  are  not  tested  may 
fail  to  hold.  However,  some  recent  results  by  Arbuckle  and  Larimer  (1976) 
indicate  that  the  double  cancellation  requirement  becomes  stronger  as  the 
size  of  the  factorial  design  increases.  For  large  factorial  designs  (e.g., 
5  x  5,  6  x  6) ,  the  study  showed  that  the  chance  probability  of  observing 
a  constraints  matrix  which  satisfies  both  first-  and  second-order  cancel¬ 
lation  is  very  small.  Since  the  ICM  algorithm  may  be  most  useful  with 
large  factorial  designs,  and  since  the  computational  requirements  in  test¬ 
ing  third-  and  fourth-order  cancellation  were  found  to  require  large 
amounts  of  computer  time,  only  the  first-  and  second-order  cancellation 
tests  were  implemented  in  the  present  version  of  the  algorithm.  The  test 
for  sufficiency  of  the  constraints  matrix  for  the  AIM  may  be  performed  by 
a  linear  programming  subroutine. 

The  antecedent  constraints  ascertained  by  querying  the  subject  may 
be  incorrect  when  random  error  is  present  in  an  experiment.  Any  conclu¬ 
sion  constraints  derived  from  cancellation  conditions  based  on  an  incor¬ 
rect  antecedent  constraint  will  also  be  incorrect.  The  ICM  algorithm 
must  include  a  mechanism  to  detect  ordinal  inconsistencies  in  the  con¬ 
straints  matrix  and  to  rectify  those  discrepancies  from  the  AIM  when  they 
are  discovered. 

The  mechanism  for  detecting  ordinal  inconsistencies  involves  simply 
checking  the  conclusion  constraint  for  each  possible  set  of  antecedent 
constraints  generated  on  the  basis  of  Scott's  (1964)  cancellation  axiom. 
When  a  particular  set  of  antecedent  constraints  implies  a  conclusion  con¬ 
straint,  the  following  logic  is  used: 

1.  If  no  conclusion  constraint  has  been  entered  in  the  constraints 
matrix,  the  implied  constraint  is  entered  and  the  algorithm 
proceeds  to  test  the  next  set  of  antecedent  constraints. 

2.  If  the  correct  conclusion  constraint  has  been  entered  in  the 
constraints  matrix,  the  constraints  are  consistent  and  the 
algorithm  proceeds  to  test  the  next  set  of  antedecent 
constraints . 
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3.  If  an  incorrect  conclusion  constraint  has  been  entered  in  the 
constraints  matrix,  an  ordinal  inconsistency  has  been  detected. 

An  error-correction  strategy  must  be  adopted  to  rectify  the 
discrepancy  from  the  AIM.  An  ordinal  inconsistency  detected 
in  this  manner  may  stem  from  one  of  two  sources:  (1)  one  or 
more  of  the  antecedent  constraints  may  be  incorrect;  (2)  the 
conclusion  constraint  may  be  incorrect.  For  the  purposes  of 
this  study,  a  simple  error-correction  strategy  was  adopted. 

Each  antecedent  constraint  was  queried  in  turn  until  a  con¬ 
straint  different  from  the  table  entry  was  obtained.  If  all 
the  antecedent  constraints  were  found  to  correspond  to  the 
table  entries,  the  table  value  for  the  conclusion  constraint 
was  changed  to  the  derived  value.  Since  any  constraints  which 
were  altered  may  have  been  used  in  previously  tested  conditions, 
it  was  necessary  to  restart  the  cancellation  condition  testing 
process  from  the  beginning  each  time  an  ordinal  inconsistency 
was  corrected. 

The  ICM  algorithm  logic  described  above  was  implemented  on  a  CDC  3300 
computer.  A  listing  of  the  FORTRAN  program  is  provided  in  the  appendix. 
Some  internal  program  documentation  is  provided  by  means  of  comment  state¬ 
ments.  Program  MAIN  is  the  driver  program  which  initializes  the  algorithm 
and  successively  calls  up  the  first-  and  second-order  cancellation  test 
subroutines  CAN1  and  CAN2 ,  respectively.  Subprograms  ASK,  OUTPUT,  INDEX, 
and  XNORM  perform  a  variety  of  utility  functions  as  indicated  by  comments 
in  the  listing.  Since  the  linear  programming  subroutine  LPSUB  was  taken 
directly  from  Davisson  (1972) ,  the  listing  is  not  included  in  the  appendix 
The  results  of  testing  the  first-  and  second-stages  of  the  algorithm  pro¬ 
duced  discouraging  results,  so  the  third-stage  of  the  algorithm  repre¬ 
sented  by  LPSUB  was  not  fully  implemented. 


ALGORITHM  EVALUATION 

Error-Free  Data 

The  purpose  of  testing  the  ICM  algorithm  in  error-free  data  was  to 
determine  the  degree  of  success  in  minimizing  the  redundancy  problem. 

In  a  complete  pair-comparison  experimental  design  with  N  composite  stimuli 
M  =  N(N-l)/2  nontrivial  pair-comparisons  are  made  to  generate  a  complete 
constraints  matrix.  The  degree  of  success  in  minimizing  the  redundancy 
problem  may  be  measured  by  contrasting  the  number  of  queries  of  the  sub¬ 
ject  required  by  the  ICM  algorithm  to  generate  a  complete  constraints 
matrix  in  error-free  data  with  the  theoretical  maximum  number  M. 

A  5  x  5  factorial  design  was  chosen  for  the  test.  The  25  composite 
stimuli  yield  a  constraints  matrix  with  25(25-l)/2  =  300  constraints. 

The  five  values  of  each  component  stimulus  were  generated  in  accordance 
with  a  random-effects  model  by  sampling  randomly  from  independent  normal 
distributions  with  means  equal  to  zero  and  variances  equal  to  unity.  The 
25  values  of  composite  stimuli  were  generated  in  accordance  with  a 


strict  additive-independence  model  by  summing  appropriate  component  stimu¬ 
lus  scale  values.  The  resulting  5x5  matrix  of  composite  stimuli  is  the 
PSI  matrix  in  the  ICM  algorithm  computer  program  listing.  The  PSI  matrix 
generated  for  the  error-free  data  test  is  shown  in  Figure  1. 

PSI  MATRIX 


P 

U 

H 

S 

T 

A 

1.3290 

•  8  1 4  l 

.5348 

.323* 

•  60g3 

ti 

-0.5b55 

-l.OBO* 

-1.3597 

-1.5707 

1.2062 

c 

1.9932 

1 .4783 

1.1990 

.  9b8  ) 

1.2725 

D 

1.0116 

•  496  7 

.2174 

.  0064 

.2909 

t 

.0796 

-;) .  4353 

-0.7146 

-0.9256 

0.6411 

Figure  1. 

PSI  matrix 

of  composite 

scale  values. 

In  the  first  stage  of  the  ICM  algorithm,  the  test  of  first-order 
cancellation  required  only  20  queries  in  order  to  fill  in  100  constraints 
in  the  matrix.  These  100  constraints  are  shown  as  1  or  -1  above  the  main 
diagonal  in  the  CONSTRAINTS  matrix  in  Figure  2.  The  table  value  of  9  in¬ 
dicates  that  a  constraint  was  neither  queried  nor  derived  in  the  first- 
order  cancellation  stage.  The  20  constraints  which  were  queried  in  the 
first-order  cancellation  stage  are  shown  as  1  in  the  TIMES  matrix  in 
Figure  3.  The  table  value  of  0  indicates  that  a  constraint  was  not 
queried  in  the  first-order  cancellation  stage. 

In  the  second  stage  of  the  ICM  algorithm,  the  test  of  second-order 
cancellation  required  149  queries  in  order  to  fill  in  the  remaining  200 
constraints.  The  complete  CONSTRAINTS  matrix  is  shown  in  Figure  4.  The 
149  queries  required  by  second-order  cancellation  plus  the  20  queries  re¬ 
quired  by  first-order  cancellation  are  shown  as  1  in  the  TIMES  matrix 
in  Figure  5.  The  table  value  of  0  indicates  that  a  constraint  was  not 
queried  in  either  cancellation  stage. 

In  total,  169  queries  were  required  by  the  ICM  algorithm  to  fill  in 
all  300  constraints  for  a  reduction  of  approximately  44%.  Although  the 
savings  were  substantial,  more  than  three  minutes  of  CDC  3300  computer 
time  were  required  to  compile,  load,  and  execute  the  program  to  implement 
the  first  two  stages  of  the  algorithm.  If  the  third  stage  of  the  algo¬ 
rithm  (r.e.,  linear  programming  for  scaling  solutions)  had  been  imple¬ 
mented  for  this  test,  the  computer  time  and  core  memory  requirements  would 
have  been  much  more  substantial.  Thus,  the  computational  requirements  of 
the  ICM  algorithm  based  on  a  direct  application  of  Scott's  (1964)  can¬ 
cellation  condition  may  be  considered  rather  excessive.  A  more  sophisticated 
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Figure  5.  TIMES  matrix  after  first  order  cancellation  in 
error-free  data. 
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Figure  U.  CONSTRAINTS  matrix  after  second-order  cancellation 
in  error-free  data. 
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mathematical  approach  seems  required  to  provide  a  more  efficient  solution 
to  the  redundancy  problem.  The  principal  disadvantage  of  the  present  ap¬ 
proach  seems  to  be  the  inherent  rigidity  in  constructing  the  second-order 
cancellation  tests  based  on  specific  sequences  for  testing  all  a,  b,  c, 
in  A  and  all  p,  q,  r  in  P.  In  order  to  further  reduce  the  number  of 
queries  required  in  the  second-order  cancellation  stage  of  the  algorithm, 
it  seems  necessary  to  abandon  thi s  rigid  procedure  in  favor  of  an  adap¬ 
tive  procedure  based  on  the  nature  of  all  the  known  constraints.  Just 
how  to  accomplish  this  goal  in  the  present  context  of  Scott's  (1964) 
cancellation  condition  is  not  clear.  Perhaps  an  adaptation  of  the  gen¬ 
eral  algorithm  of  McClelland  and  Coombs  (1975)  may  be  feasible  for  this 
purpose  of  interactive  conjoint  measurement. 


Data  With  Random  Error 


The  purpose  of  testing  the  ICM  algorithm  in  data  with  random  error 
was  to  determine  the  degree  of  success  in  minimizing  the  problem  of  ran¬ 
dom  error.  This  degree  of  success  may  be  assessed  in  terms  of  (1)  the 
number  of  additional  queries  required  by  the  ICM  algorithm  to  resolve 
detected  discrepancies  from  the  AIM,  ind  (2)  the  extent  to  which  the 
estimated  constraints  matrix  corresponds  to  the  true  constraints  matrix 
(i.e.,  the  extent  to  which  the  obtained  measurement  structure  approxi¬ 
mates  the  true  latent  measurement  structure) . 

The  5x5  PSI  matrix  of  composite  choice  alternative  scale  values 
generated  for  the  test  of  the  ICM  algorithm  in  error-free  data  was  also 
used  for  the  test  of  the  algorithm  in  data  with  random  error.  Whenever 
the  algorithm  queried  the  simulated  "subject,"  the  true  absolute  differ¬ 
ence  in  composite  choice  alternative  scale  values  was  perturbed  by  add¬ 
ing  an  error  component  prior  to  ascertaining  the  preference  relation. 

The  error  components  were  sampled  independently  from  a  normal  distribu¬ 
tion  with  mean  equal  to  zero  and  variance  equal  to  o^.  The  error  vari¬ 
ance  was  set  at  a  moderate  value  of  .25. 

The  results  of  this  test  showed  that  a  total  of  200  queries  were 
required  to  generate  a  complete  constraints  matrix  which  satisfied  both 
first-  and  second-order  cancellation  conditions.  Thus,  the  44%  reduction 
obtained  in  error-free  data  was  decreased  to  33%  reduction  in  data  with 
a  moderate  amount  of  random  error.  The  number  of  queries  made  for  each 
pair  of  composite  choice  alternatives  is  shown  in  the  TIMES  matrix  in 
Figure  6.  Note  that  comparisons  (a,r)  versus  (a,s)  and  (a,q)  versus  (a,t) 
were  made  nin  :  times  and  eight  times,  respectively.  The  large  numbers 
of  queries  required  for  these  comparisons  may  be  attributed  to  the  smnll 
absolute  differences  in  scale  values  (approximately  .21)  which  make  these 
comparisons  especially  sensitive  to  random  error  (standard  deviation  .50). 
The  fact  that  the  algorithm  made  repeated  queries  of  these  comparisons 
rather  than  queries  of  equivalent  comparisons  [e.g. ,  (b,r)  versus  (b,s) ; 
(b,q)  versus  (b,t) ]  indicates  another  deficien  :y  of  this  algorithm  which 
is  due  to  the  inherent  rigidity  of  constructing  cancellation  conditions 
based  on  specific  sequences  of  elements  in  the  component  stimulus  sets. 


15 


TIMES  matrix 


lb 


The  estimated  CONSTRAINTS  matrix  obtained  in  this  test  is  shown 
in  Figure  7.  Note  that  this  constraints  matrix  corresponds  to  the  true 
constraints  matrix  shown  in  Figure  4  with  the  exception  of  three  con¬ 
straints:  (a,r)  versus  (d,q) ,  (b,p)  versus  (e,q) ,  and  (c,r)  versus 

(d,p).  Since  perfect  correspondence  was  not  obtained,  these  results 
imply  a  need  to  test  higher-order  cancellation  conditions  prior  to 
passing  the  constraints  matrix  to  the  linear  programming  third  stage 
of  the  algorithm.  More  than  six  minutes  of  CDC  3300  computer  time  were 
required  to  compile,  load,  and  execute  the  program  for  this  test  with 
random  error.  Again,  the  computational  requirements  of  the  ICM  algo¬ 
rithm  based  on  a  direct  application  of  Scott's  (1964)  cancellation  con¬ 
dition  seem  excessive.  A  more  computationally  efficient  algorithm  seems 
required  to  serve  as  a  basis  for  computerized  adaptive  model  testing. 


CONCLUSIONS 

This  study  investigated  an  application  of  recent  results  in  con¬ 
joint  measurement  theory  research  with  the  aim  of  developing  a  new 
methodology  for  (1)  modeling  a  decision  process  used  to  evaluate  pref¬ 
erences  for  complex  choice  alternatives  and  (2)  producing  measurement 
scales  for  choice  component  factors  and  composite  choice  alternatives 
based  on  the  decision  model.  An  algorithm  for  interactive  conjoint 
measurement  (ICM)  was  developed  to  minimize  the  problems  of  redundancy 
and  random  error  in  testing  the  additive- independence  model  (AIM)  in 
pair-comparisons  designs  with  fallible  data.  No  provisions  were  made 
for  handling  the  systematic  error  problem  or  for  accommodating  more  than 
two  choice  component  factors. 

The  results  showed  that  a  reduction  of  approximately  44%  in  the 
number  of  pair-comparisons  necessary  to  determine  all  model  constraints 
was  possible  in  a  5  x  5  factorial  design  with  error-free  data.  This  re¬ 
duction  was  decreased  to  33%  when  a  moderate  amount  of  random  error  was 
introduced.  Although  only  first-  and  second-order  cancellation  condi¬ 
tions  were  built  into  the  algorithm  for  this  test,  the  test  in  error-free 
data  required  more  than  three  minutes  of  computer  time  and  the  test  in 
data  with  a  moderate  amount  of  random  error  required  more  than  6  minutes 
of  computer  time.  The  main  conclusion  from  the  study  was  that  the  ICM 
algorithm  based  on  a  direct  application  of  Scott's  (1964)  cancellation 
axiom  performs  rather  poorly,  and  that  a  more  computationally  efficient 
algorithm  seems  required  to  serve  as  a  basis  for  computerized  adaptive 
model  testing. 
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CONSTRAINTS  MATRIX 

AP  AO  AR  AS  AT  8P  So  OR  MS  BT  CP  CO  CS  CS  CT  OP  00  OR  OS  OT  EP  EO  ER  ES  ET 

~if>  a  i  I  I  i  i  i  i  1  i  -l  -i  T  i  i  i  i  j  i  i  I  1  i  FT 

ao  -i  o  i  t  i  l  i  i  1  i  -l  -i  -i'  -i  -l  -i  l  i  i  i  i  i  T  i  T 
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MSFOHTRAN  (4,3)  /  MSOS  5,1  07/25/79  PAGE  001 

PROGRAM  main 

OIMENSION  RO*A{5> ,C0LP(5) *IJtH(2) 

Integer  conmat, times, sumit 
Integer  a,p»flag 
character  aa»pp» AST 

COMMON  /OATA/  NAtNAMl ,NAM2,NAM3,naM4*NP»NPM1 ,NPM2,NPM3,NPma» 
1A(5).P(5) ,FLAG,AA(5> tPP (5) ,AST,pSI <St5> »CONMAT (625) »TIMES<625)  tSE 
2*IX 


1 G 1 


1  02 


103 

104 


INITIAL 

I ZAT IONS 

AA(1)  = 

21B 

AA(2)  a 

22B 

AA ( 3)  = 

23B 

AA (4)  = 

?4B 

AA (5)  = 

258 

PP(1)  = 

478 

PP (2)  = 

50B 

PP (3)  = 

51B 

PP  <  4 )  = 

62B 

PP(5)  a 

63B 

AST  =  543 

IX  x  4321567 

SA  =  1. 

SP  *  1. 

SF  »  0, 

NA  =  5 

NP  a  5 

N  AM  1  * 

NA  - 

1 

NAM2  a 

N  A  - 

2 

NAM3  a 

NA  - 

3 

NAM4  x 

NA  - 

4 

NPM 1  a 

NS  - 

I 

NPM2  x 

N3  - 

2 

NPM3  * 

N3  - 

3 

NPM4  x 

NS  - 

4 

MAXIT  a 

ISO 

SOM  IT  a 

0 

flag  = 

l) 

DO  lol 

Ial,2 

ITER(I) 

a  0 

CONTINUE 

I  X  (NA 

*NP>**2 

DO  102 

J*1,I 

CONMAT ( 

J)  = 

9 

times (  j>  *  o 
continue 

00  1()3  1  =  1  »NA 

00  1  l)J  J=1»NP 

IND  =  IN3EX(I*J»I*J) 

CONMAT (I  NO)  ■  U 

CONTINUE 

DO  104  1*1, NA 

ROMA(l)  s  XNORM ( lX,0« *5A) 

CONTINUE 

DO  106  J= 1 , NP 


Li-'vvrxrr: 
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COLP(J)  *  XNORMUX.O..SP) 

105  CONTINUE 

00  106  Is) .NA 
DO  106  Jsl.NP 

PSI<I»J>  *  ROWA(I)  ♦  COLP(J) 

106  CONTINUE 
C 

c  test  cancellation  axioms 

c . 

201  call  cani 

IF  (FLAG  .NE.  0)  GO  TO  401 

CALL  OUTPUT  (NA.NP.SUMIT.ITER.PS! .CONMAT, TIMES. AA.PP) 
call  CaN2 

IF  (FLAG  .NE.  0)  GO  TO  401 

301  call  output  (na.np.sumit.iter.psi.conmat.times.aa.pp) 

GO  TO  501 

401  SUMlT  S  SUM IT  «  1 

ITER(FlAG)  •  I  TEH (FLAG)  ♦  1 
IF  (SUMlT  ,GT.  MAXIT)  GO  TO  301 
flag  ■  o 

GO  TO  201 
501  CONTINUE 
STOP 

end 
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subroutine  cani 
integer  conmat. times 
integer  ai.A2,a3»A4»pi»p2»P3*P4 
integer  a.p.flag 
character  aa.pp.ast 

COMMON  /OATA/  NA»NAMl»NAM2»NAM3,NAM4tNP»NPMl,NPM2,NPM3tNPM4» 

1 A (5) ,P(5) .FLAG.AA(S) »PP(5> .AST.PSI I5t5) »C0NMAT<625) .T1MES(625>  »SE 
2. IX 

EQUIVALENCE  ( A  ( 1 ) «IA) *  (A(2) ,JA> , (A(3) »KA) 

EQUIVALENCE  <P(1> .IP) • (P(2) .JP) , <P(3>  »KP) 

CANI  TESTS  FIRST  ORDER  CANCELLATION 

IF  (NA  .LT.  2  .OR.  NP  .LT.  2)  GO  TO  9999 

IN  FIRST  ORDER  CANCELLATION.  THE  SEQUENCE  LOGIC  IN  SCOTTFS  THEOREM 

may  be  replaced  by  subset  logic  with  no  loss  of  generality  and  with 
substantial  savings  in  computer  time. 

TEST  Each  SUBSET  OF  2  ELEMENTS  IN  A 

IA  =  0 
IA  =  IA  ♦  1 

IF  (IA  .EU.  NA)  GO  TO  9999 
JA  *  I A 
JA  =  JA  ♦  l 

IF  (JA  • 3T •  NA)  GO  TO  1 

«  IF  (IA  .EQ.  JA)  GO  TO  2 

TEST  tACH  SUBSET  OF  2  ELEMENTS  IN  P 

) 

IP  »  0 
IP  a  IP  ♦  1 

IF  (IP  .EG.  NP)  GO  TO  2 
JP  «  IP 
JP  a  JP  ♦  1 

IF  (JP  .GT.  NP)  GO  TO  3 

IF  (IP  .EQ.  JP)  GO  TO  4 

TEST  Each  PERMUTATION  OF  ELEMENTS  IN  SUBSET  OF  A 
00  102  Ial  .2 

00  101  Ja-1.2 

IF  (I  .EQ.  J)  GO  TO  101 

TEST  EACH  (NONTRIVIAL)  PERMUTATION  OF  ELEMENTS  in  SUBSET  OF  P 

II  a  J 
JJ  a  1 

TEST  FIRST  ORDER  CANCELLATION' 

A 1  a  lA 
A2  a  A ( I ) 

A3  a  A ( J) 

A4  a  JA  23 


MSfORTRAN  (4.3)  /  MSOS  5.1  07/25/79 

PI  a  IP 
P?  *  P(II) 

P3  *  P(JJ) 

P4  *  JP 

IND1  »  lV0EXUl.PltA2.P2) 

10R1  *  CONMAT ( INOl ) 

IF  <I0R1  ,EU.  9) 

1  CALL  ASK  (IOR1.A1.P1.A2.P2) 

IND2  *  IVDEX(A3«P3.A4.P4) 

10R2  *  CONMAT (1NU2) 

IF  (I0R2  .NE.  9)  00  TO  51 
CONMaT ( I VD2)  a  10R1 
IND  =  INDEX (A4.P4.A3.P3) 

CONMAT (IvO)  *  -10R1 
GO  TO  101 

51  IF  (I0R2  ,EU.  I0K1)  00  TO  101 
flag  *  1 

WRITE  (61.1001)  AA(A1).PP(P1).AST.AA(A2).PP(P2). 

1  AA (A3) *PP(P3) .AST  »AA (A4) .PP(P4) 

1001  FORMAT  (//(1X.2A1.1X.A1.1X.2A1)) 

WRITE  (61.1002)  10R1.I0R2 

1002  FORMAT  (  /IX. 215) 

call  ask  uor.ai.pi.a2.p2) 

IF  (IOR  ,NE.  IORi )  60  TO  9999 

CONMAT ( I V02)  »  I0K1 

IND  s  INDEXU4.P4.A3.P3) 

CONMAT (I  VO)  =  -IORI 
GO  TO  9999 

101  CONTINUE 

102  CONTINUE 
00  TO  4 

9999  RETURN 
END 
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subroutine  can2 
integer  conmat, times 

INTEGER  A] »A2.A3*AAtA5«A6.Pi»P2,P3tP4fP5.P6 
integer  a. p. flag 
Character  aa.pp.ast 

common  /DATA/  NA.NAMl,NAM2.NAM3,NAM4»NPfNPMl,NPM2,NPM3«NPMA» 

1 A (5 > »P (5) ,FLAG.AA(5> *PP (5) .AST.PSI (5*5) .CONMAT (625) .TIMES (625)  »SE 
2. IX 

EQUIVALENCE  ( A ( 1 > ,IA) . (A (2) ,  JA) , (A (3) »KA) 

EQUIVALENCE  (P(1).IP).<P<2).JP) , (P(3) »KP> 

can?  tests  seconu  order  cancellation 

IF  (NA  .LT.  3  .OR.  NP  .LT.  3)  GO  TO  9999 

test  Each  (nonrepeating)  sequence  of  3  elements  In  a 

I A  3  0 
IA  3  IA  ♦  1 

IF  (IA  .ST.  NA)  GO  TO  9999 
JA  3  0 
JA  3  Ja  ♦  1 

IF  (JA  .GT.  NA)  GO  TO  1 
IF  (IA  .EO.  JA)  GO  TO  2 

KA  3  U 

KA  3  KA  ♦  1 

IF  (KA  .GT.  NA)  GO  TO  2 

IF  (IA  .£Q.  KA  .OR.  JA  .EO.  KA)  GO  TO  3 

TEST  EaCH  (NONREPEATING)  SEQUENCE  OF  3  ELEMENTS  IN  P 

IP  3  0 
IP  3  IP  ♦  1 

IF  (IP  .ST.  NP)  GO  TO  3 
JP  3  0 

jp  *  JP  ♦  1 

IF  (JP  .GT.  NP)  GO  TO  4 
IF  (IP  .EQ.  JP)  GO  TO  S 
KP  3  0 
KP  ■  KP  ♦  J 

IF  (KP  .GT.  NP)  GO  TO  5 

IF  (IP  .EQ.  KP  .UR.  JP  .EQ.  KP)  GO  TO  6 

TEST  Each  PERMUTATION  of  elements  In  SEQUENCE  OF  a 

00  106  I«1 f 3 
00  105  J*1 .3 
IF  (I  .EQ.  J)  GO  TO  105 
00  104  K*l,3 

IF  (I  .EQ.  K  .OR.  J  »Eu.  K>  30  TO  104 

TEST  EACH  (NONTRIVIAL)  PERMUTATION  OF  ELEMENTS  IN  SEQUENCE  Of  P 
00  103  1 1»  1  *3 

IF  (I  .EQ.  1  .AND.  II  .EQ.  1)  GO  TO  103 
00  102  JJ»1 .3 

25 
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IF 

<11  .EQ.  JJ)  GO 

TO 

102 

IF 

(J  .EU.  2  .AND. 

JJ 

.EQ. 

2)  GO  TO  102 

DO 

101  A<*1  .3 

IF 

(11  .EQ.  KA  .OR 

.  JJ 

.EQ. 

AA)  GO  TO  101 

IF 

(K  .EU.  3  .AND. 

AA 

.EQ. 

3)  GO  TO  101 

IF 

(K  .EU.  1  .AND. 

AA 

.EQ. 

1  .AND.  I  *EQ. 

3 

.AND. 

11  .EQ. 

3) 

1  < 

SO  TO  101 

IF 

(A  .EU.  2  .AND. 

AA 

.EQ. 

2  .AND.  J  .EQ. 

3 

.AND. 

JJ  .EQ. 

3) 

1  GO  TO  101 

test  second  order  cancellation 


A 1  r  lA 
A2  =  A ( I ) 

A3  *  <JA 
A4  *  A ( J) 

A  5  m  A  (  K ) 

A6  =  AA 
Pi  a  IP 
P2  *  P(II) 

P3  *  OP 
P4  =  P(JJ) 

P5  =  P(KK) 

P6  »  AP 

INDI  =  INDEX(A1»P1»A2,P2) 

IOR1  *  CONMAT(INUl) 

IF  ( I OR 1  .ED.  9) 

1  CALL  ASA  <I0RltAl»PltA2tP2) 

IND?  »  I NOEX ( A3*P3«A4»P4) 

I0R2  =  CONMAT ( INU2 ) 

IF  (I0R2  .EU.  9) 

1  CALL  ASA  (I0R2»A3»P3fA4tP4> 

IF  (IOR2  ,NE.  IOR1)  GO  TO  10 1 
IND3  ®  l90EX(A5»P5»Ab»Pb) 

I0R3  *  CdnmAT(IND3) 

IF  (IOR3  ,NE.  9)  GO  TO  51 
CONMAT (1ND3)  *  IOR2 
IND  *  INDEX ( A6*Pb* A5fP5) 

CONMAT (ISO)  =  -IOR2 
GO  TO  101 
51  IF  (IOR3  .EU.  I0H2)  GO  TO  101 
flag  *  2 

WRITE  (61.1001)  AA(A1)»PP(P1).AST»AA(A2),PP(P2). 

1  AA ( A3) »PP  <P3) . AST  »AA ( A4 ) »PP(P4) » 

2  AA ( A5) »PP(P5>  »AST*AA(A6>  »PP(P6> 

1001  FORMAT  (//(1X.2A1.1X.A1.1X.2A1)) 

WRITE  (61.1002)  10R1.I0R2.I0R3 

1002  FORMAT  (/IX. 315) 

CALL  ASA  (I0R.A1.P1.A2.P2) 

IF  (IOR  , NE.  IORl)  GO  TO  9999 
CALL  ASA  (I0R.A3.P3.A4.P4) 

IF  (IOR  ,NE.  I0R2)  GO  TO  9999 

CONMAT (1903)  *  IOR2 

INO  «  INDEX (A6.Pb.A5.P5) 

CONMAT (190)  a  -10R2 

GO  TO  9999  26 
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101 

CONTINUE 

102 

CONTINUE 

103 

CONTINUE 

104 

CONTINUE 

105 

CONTINUE 

106 

CONTINUE 
GO  TO  6 

9999 

RETURN 

END 
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subroutine  ask  (1or*i.j*k,d 
integer  a.p.flag 
integer  conmat. times 

CHARACTER  AA»PP»AST 

COMMON  /DATA/  NA *NAM1 « NAM2 «NAM3 ,NAM* *NP»NPM1 , NPM2 ,NPM3 ,NPM* i 
1A<5) ,P<5) .FLAG* AA (5) .PP(5> .AST.PSI (5*5) .CCNMAT (625) » TIMES (625) *SE 
2*  IX 

ASK  QUERIES  THE  SUBJECT  CONCERNING  THE  PREFERENCE  RELATION 
FOR  STIMULUS  (I*J>  RELATIVE  TO  SIMULUS  (K»L). 

PREF  «  PSI(I«J)  -  PSI(K.L) 

PREF  =  PREF  ♦  XNORM(lX*0.0*SE) 

IF  (PREF)  1,2.3 

I OR  *  -1 

GO  TO  4 

IOR  =  0 

GO  TO  4 

IOR  =  1 

CONTINUE 

INO  a  IN0EX(I. J.K.L) 

CONMAT (INO)  *  IOR 
TIMES(IND)  a  TIMES ( INO)  ♦  1 
IND  =  INDEX(K,L*I»J) 

CONMAT ( INU)  a  -IUR 
RETURN 

End 
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SUBROUTINE  OUTPUT  (NA.NP. SUMl T  *IT£R»PSI .CONMaT  , TIMES. AA»PP) 
dimension  ITER (2) ,PSI (S*5) 

INTEGER  SUMIT, CONMAT (625) « TIMES (625) »TEMP(5*5) 
character  aa(5) »pp(S) 
c 

C  OUTPUT  ITERATIONS 

C 

WRITE  (61,1001)  SuMlT,  ( ITER ( I >, 1.1,2) 

1001  FORMAT  (1H1//11H  ITERATIONS 

1  //6H  SUMIT.2X.I3 

2  //6H  ITER1.2X.I3 

3  //6H  ITER2.2X.l3) 

C 

C  OUTPUT  CONSTRAINTS  MATRIX 

C 

WRITE  (61.1002)  ( (AA(I) »PP( J) » j.l.NP) ,I»l,NA) 

1002  FORMAT  <1H1//1*H  CONSTRAINTS  MATRIX//6X.25 (2a1 ,2X) ) 

DO  l 0 1  1=1, NA 

DO  1 0 1  J=1»NP 

DO  1  Ksl.NA 

DO  1  L=1,NP 

IND  =  INOEXd.J.K.L) 

TEMP(K.L)  *  CONMAT(IND) 

1  'CONTINUE 

WRITE  (61.1003)  AA(I) ,PP( J) , ( (TEMP(K.L) »L*1.NP) .Krl.NA) 

1003  FORMAT  (/2X.2A1.2X, 25(12. 2X)) 

101  CONTINUE 
C 

c  output  times  matrix 
c 

WRITE  (61,1004)  ( (AA(I) ,PP( J) .U.l.NP) ,I=1,NA) 

1004  FORMAT  (1H1//13H  TIMES  MATri X//*X,25 (2Al , 2X> ) 

DO  102  1=1, NA 

DO  102  J=1,NP 

DO  2  K=i,NA 

DO  2  L*1 ,NP 

IND  =  INDEX ( I , J.K.L) 

TEMP(K.L)  =  TIMES ( IND) 

?  CONTINUE 

WRITE  (61,1003)  AA(I) ,PH(J) ,< (TEMP(K.L) ,L»I»NP) ,X=1.NA) 

102  CONTINUE 
C 

C  OUTPUT  PSI  MATRIX 

C 

WRITE  (61,1005)  (PP( J) .J=1,NP) 

1005  FORMAT  ( 1  HI// 1 1 H  PSI  MATRIX//6X,5(5X, A1.4X) ) 

DO  1 1)3  1  =  1,  NA 

WRITE  (61,1006)  AA(I) , (PSI (I.J) ,J=1 »NP) 

100*  FORMAT  (/3X.A1.2X.5F10.4) 

103  CONTINUE 
RFTURN 

En() 
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function  inoex<i*j*k»u 

COMMON  /DATA/  M»10UM(4)»N 

index  computes  the  location  of  element  <i.j«k,d  in  a  linear 

Fouw-OIMENSIONAL  ARRAY  OF  SIZE  (M»N,M,N). 

INDEX  »  I  ♦  ( J-l ) #M  ♦  (K-1)*M»N  ♦  (L-l ) *M4N*M 
RETURN 
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function  xnorm<ix.xm.xsi 

xnorm  computes  a  normally  distributed  random  variable  with 

MEAN  XM  AND  STANDARD  DEVIATION  xS. 

XNORM  =  0. 

DO  101  1=1.12 
IY  =  IX  *  4093 
IF  ( I Y  >  1.2.2 

1  IY  =  IY  ♦  8388607  ♦  1 

2  YFL  =  IY 

YFL  =  YFl  *  0. 119209JE-06 
IX  =  IY 

XNORM  =  XNORM  ♦  YFL 
101  CONTINUE 

XNORM  a  (XNORM  -  b.)  *  XS  ♦  XM 
RETURN 

End 
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